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Project Change began in 1994 when all of the sites were still in planning or 
early action phases. Findings from the summative evaluation will be available 
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available previously. It covers: (l) development of the Project Change model 

and theory of change; (2) evaluation approach and challenges; (3) early 
findings; and (4) next steps in evaluation. Some of the special challenges of 
anti-racism work have been identified. These include the fact that people 
want to know whether the work is effective long before tangible results are 
likely to be produced. Because it is not clear what it will take to solve the 
problem of racism, it is hard to use markers of progress to predict eventual 
success from early results. Evaluation findings do show that communities have 
benefited from Project Change in some tangible ways. The policies of lending 
institutions have changed to make new funds available to poor people. Project 
sites have raised awareness about hate crimes, and communities are beginning 
to institutionalize their Project Change work so that it will last beyond the 
initial funding. Some unintended benefits have come to participants in terms 
of new skills and a higher degree of awareness. Some social policy benefits 
have also been apparent as lessons generated by project management, the 
communities, and the evaluation have helped stimulate anti- racism activities 
by other funders and in other communities. (SLD) 
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EXECUTIVE SUMMARY 



The Levi Strauss Foundation (LSF) asked the Center for Assessment and Policy Development 
(CAPD) to evaluate its Project Change initiative. Project Change is a community-driven anti- 
racism initiative operating in four communities - Albuquerque, NM; El Paso, TX; Knoxville, TN; 
and Valdosta, GA. Albuquerque, El Paso and Valdosta began planning in 1991; Knoxville 
joined Project Change and began planning in 1993. In 1997, a national Project Change office 
was formed by a gift from LSF to the Tides Center. The national office provides an institutional 
base for Project Change, and a place from which additional anti-racism work can be developed 
and implemented. 

We began our formative evaluation of Project Change in 1994, when all of the sites were still in 
planning or early action phases. Formative evaluation continued through 1998. Numerous 
internal reports for the sites, staff and Board of LSF have been produced. In addition, early 
findings and lessons from evaluation have been published and distributed widely. Lessons on 
planning an anti-racism initiative were published in 1996; lessons from early implementation 
were published in 1999. Findings from summative evaluation will be available in 2002. 

The purpose of this research brief is to provide more technical information than has been 
available previously. It covers: 1) development of the Project Change model and theory of 
change; 2) evaluation approach and challenges; 3) early findings; and 4) next steps in 
evaluation. We hope this brief will help other communities, funders, policy makers and 
researchers figure out good ways to tell their stories, track their progress and document the 
results of their anti-racism work. 

Major challenges for evaluators, and early findings about Project Change, are summarized 
below: 

Challenges for evaluation 

Besides the usual technical issues related to evaluation of multi-year, multi-site community-wide 
initiatives, evaluation of anti-racism work poses some special challenges: 

• For a variety of reasons (impatience with things as they are; concern that scarce 
resources be well-used; perceived riskiness of the work) people want to know 
whether or not the work is effective long before tangible results are likely to be 

produced; 

• As a field and a nation, we have never solved the problem of racism. So we don 7 
have a clear picture of what it will take to do so. This makes it hard to use 
markers of progress to predict eventual success from early results. 

• There is racism in evaluation, as in other spheres. A key question is what 
constitutes success and who says so. 
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Communities have benefitted in tangible ways from Project Change. 

- Fair lending strategies have changed policies and practices of lending 
institutions; and made new funds available to poor people and people of 
color in at least two of the sites. 

- Sites have raised awareness about hate crimes, and in at least two of the 
sites, are involved in training law enforcement officials in different ways 
of identifying and prosecuting them. 

- Communities are beginning to institutionalize their Project Change work 
so it will last beyond the period of LSF funding. For example, 

Albuquerque Project Change has created the Albuquerque Project 
Change Fair Lending Center at the Public Law Center of the University of 
New Mexico. APC FLC is fully staffed and trains communities to take 
advantage of Fair Lending laws and regulations to increase access to 
capital for people of color, as APC did in coalition with other advocates 
and lending institutions. 

- While Project Change was still planning and implementing its strategies, 
it was already producing benefits from other activities. Unintended 
benefits came from responding well to opportunities for anti-racist action; 
serving as a watchdog organizations or coalition; and from activities that 
leaders of Project Change took, outside their Project Change work, that 
they attributed to renewed passion and specific skills acquired through 
their participation in Project Change. 

- Project Change has been of benefit from a social policy perspective, as 
well as to participating communities. Lessons generated by project 
management, the communities and evaluation have helped to stimulate 
and inform anti-racism action by other funders and in other communities. 
The national Project Change office has produced materials - for 
example, a glossary, resource guide and toolkit with specific and practical 
information to promote better anti-racism work. 
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PROJECT CHANGE EVALUATION RESEARCH BRIEF 



The Levi Strauss Foundation commissioned an evaluation of Project Change beginning in 1994. 
The Foundation had established funding partnerships with four communities — Albuquerque, 
NM; El Paso, TX; Knoxville, TN and Valdosta, GA - to plan and implement anti-racism 
strategies. Evaluation was commissioned while the four communities were still in planning or 
early action phases. The goals of evaluation were to: 

• Capture and present data that would inform the Board of the Levi Strauss 
Foundation of the value of their investment; 

• Provide information and analysis to project management and staff of the 
Foundation to support implementation of an effective community-driven, 
multiracial anti-racism initiative, the only such foundation/community partnership 
operating in the U.S.; 

• Identify progress and challenges, and give feedback that the Project Change 
communities could use to reflect on their work and make mid-course changes, as 
appropriate; and 

• Cull and share lessons that would be useful to other communities, collaborations 
and funders who might want to replicate the Project Change model and/or 
incorporate its learnings into relevant community building and anti-racism work. 

LSF asked the Center for Assessment and Policy Development (CAPD) 1 to design and carry-out 
an evaluation that would meet these goals. Formative evaluation continued through 1998, 
covering planning and early action (or implementation) phases of the initiative. Internal reports 
for project management and the LSF Board, and feedback documents for the communities, were 
produced during each year of evaluation. Early results and learnings have also been summarized 
in two public reports that were disseminated in 1996 and 1999. LSF anticipates that a final 
round of evaluation research will be completed in 2002, to document results 10 years after 
Project Change was first begun. 



'CAPD is a non-profit research, planning and policy organization that frequently evaluates 
community/foundation partnerships aimed at improving the quality of life of children, families and neighborhoods, 
including those of color. LSF selected CAPD because of our background in those areas. We were not experts in 
anti-racism work. Like others involved with Project Change, we eventually recognized the need for additional 
training so we would have a much more informed and rigorous anti-racism vocabulary and analysis. We also had 
the opportunity over the same time period to participate in personal transformation work as evaluators of Healing the 
Heart of Diversity, a leadership development program of the Fetzer Institute for diversity professionals and social 
change agents. In our opinion, these development activities were crucial to bringing our organizational and personal 
capacities to an adequate level to do evaluation in this area. 
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The purpose of this research brief is to provide more technical information about the evaluation 
of Project Change, and its findings, than has been included in earlier public reports. As more 
diversity, multi-cultural and community building efforts recognize the importance of addressing 
personal prejudice and institutional racism as part of their work, more people have become 
interested in knowing whether or not they are being effective in those areas. In addition, as more 
anti-racism efforts are funded, there is increasing pressure to demonstrate results. This report is 
being produced to share what we have learned about assessing these kinds of efforts in hopes it 
will be useful to others. 

The rest of this brief is organized as follows: 

• Development of the Project Change model; 

• Evaluation challenges and methods; 

• Short-term markers of progress and results; and 

• Next steps. 

DEVELOPMENT OF THE PROJECT CHANGE MODEL 

Model development 

Project Change was created in 1991 as the outgrowth of a gift from Levi Strauss and Co. 
(LS&Co.) to the Levi Strauss Foundation (LSF). The Board of the Foundation challenged its 
staff to recommend an initiative that would make a substantial difference to the quality of life in 
communities where LS&Co. employees lived and worked. After considering several options, the 
Board elected to put this gift toward acknowledging and directly addressing the effects of 
racism. At that time, no other corporate or national foundation was funding community anti- 
racism work so labeled. 

The Project Change model calls for the development of community task forces to plan and 
oversee the implementation of anti-racism strategies. These strategies are designed to meet at 
least one of the following goals: 

• Dismantling institutional policies and practices that promote or encourage racial 
discrimination; 

• Easing tension between majority and minority groups, as well as inter-ethnic 
conflicts; 
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• Promoting fair representation of community diversity in the leadership of 
important community institutions; and 

• Stopping or preventing overt acts of racial and cultural prejudice. 

When this work began, the communities and LSF did not have a fully formed theory about how 
these goals could be accomplished. At the same time, they were able to, and did, draw on an 
extensive body of personal and professional experience in civic leadership, anti-racism activism, 
community organizing, community building and 20 years of LSF grantmaking. Based on this 
work and advice from a national advisory committee, they developed and implemented Project 
Change as follows: 

• LSF assessed the needs and capacities of all of the communities where LS&Co. 
had a substantial number of employees. They determined which ones had a core 
of interested multi-racial community leaders, and might accept the challenge of 
Project Change. National and regional Foundation staff and advisors spent 
several weeks in the field talking to a wide range of stakeholders in each 
community (see Lessons Learned I, How to Plan An Anti-Racism Initiative for 
more details). 

• Three communities - Albuquerque, NM; El Paso, TX; and Valdosta, GA - were 
invited to enter a planning phase. Albuquerque, EL Paso and Valdosta began 
planning in 1991. Knoxville, TN was invited to join and plan for Project Change 
in 1993. 

• The initial task forces had between 12 and 21 members, all of whom are 
volunteers. A guideline was that at least 5 1 percent of the members of each task 
force had to be people of color and at least 5 1 percent had to be women. 

LSF identified all of the initial task force members. LSF looked for individuals 
who had understanding and access to key community institutions, were 
established community leaders who together represented the full diversity of 
people in the community but who were not necessarily the traditional 
representatives of those groups. LSF’s belief was that, if the same people who 
always gathered in the community to work on social justice came together again, 
they were likely to get the same results as they always had. Thus, people on the 
initial task forces had not always worked together before Project Change, and 
some people accustomed to being the representatives of various constituencies 
were not necessarily invited to join Project Change. 

The process and criteria by which initial task force members were selected turned 
out to be a crucial element of the model, in some cases setting the stage for a fast 
start and implementation of very sophisticated and effective strategies, and in 

Center for Assessment and Policy Development 




10 



Page 4 



some cases, bringing together people with so much at stake personally and 
professionally that work never really took off until the membership turned over 
almost completely. 

Each task force selected a fiscal sponsor to receive a planning grant. They each 
participated in a two-day retreat focused on prejudice reduction and unlearning 
racism. 

LSF also funded research about the nature and effects of racism in each 
community. The research looked at potential disparities in the community 
between whites and people of color on things such as income, home ownership, 
access to capital, educational attainment, employment and business ownership; 
racial attitudes and beliefs; and qualitative information on the racial experiences 
of people living in that community. 

During the planning phase, task forces oversaw the research and participated in its 
interpretation. They used the data and their own professional and personal 
experiences to identify targets for change and strategies that could be 
implemented in an action phase of work. 

The model initially called for planning to take about six months in each 
community. However, time frames were deliberately flexible since the 
Foundation and communities both knew the model would evolve as they learned 
more about what it would take to do this work. Planning was intended to last for 
as long as it took each community to identify and coalesce its task force around 
two products: a publicly disseminated State of Race Relations Report for that 
community, based on the commissioned research, and an action plan for change 
that could lead to up to three years of LSF funding. 

Each community eventually moved into the action stage of its work. The 
planning phase ended up requiring anywhere from between a year to two and one- 
half years, depending on the “readiness” of the community to directly address 
personal prejudice and institutional racism, the composition and risk-taking 
capacity of the initial task force, the quality of paid staffing each group was able 
to identify and support, and the quality of interactions between the task force and 
the Foundation. 

Strategies are carried out by volunteers in each of the Project Change 
communities, with support from LSF as described below. A major element of the 
Project Change model is the literally thousands of volunteer hours that are 
donated to carrying out anti-racism strategies in each site, and particularly, by 
members of the Project Change task forces or Boards. 
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The Project Change model calls for communities to receive the following 
supports to implement their action strategies: 

- Grants from the Foundation to carry out strategies developed which the 
task forces and LSF agree are feasible and likely to make a difference. 
Action grants can cover things like: development and delivery of anti- 
racism training to stakeholders at targeted institutions (e.g., fair lending 
training to bankers; multi cultural curriculum training to educators, hate 
bias crime reporting and prosecution training to law enforcement 
officials); staff or consulting support to implement strategies (e.g., 
production of videos, PSAs, op-eds and other public education activities, 
organizing the community to address racial disparities in the policies of 
law enforcement, schools, banks, banking regulatory agencies); and staff 
or consulting support to work with partner institutions (e.g., to develop a 
newspaper series on race relations and the effects of racism in the 
communities, to organize anti-hate crime commissions or task forces, 
etc.). 

The original funding commitment from LSF was for planning and up to 
three one-year grants for action funding. As work proceeded, LSF 
recognized the value and need to extend funding. Funds from LSF have 
been distributed as follows: 

- an initial planning grant; 

up to three one-year grants to implement plans, based on budgets 
submitted by the sites; 

- up to two additional one-year grants as sites obtain additional 
funding and in-kind support. These transitional grants are about 
60 percent of action grants; and 

- up to two additional one-year periods of matching funding at lower 
rates. These legacy grants are about 40 percent of action grants. 

Communities are not guaranteed renewed funding. The task forces and 
LSF have negotiated these agreements year by year. 

Technical support and training on race relations, diversity and personal 
prejudice, group process and conflict resolution, media and public 
relations, coalition building, and related topics. TA and training were 
delivered in different formats such as cross-site meetings, retreats for a 
given task-force and for the wider community in community-wide forums, 
through facilitation and by participation in customized training programs. 
Some Project Change staff and task force members were also sponsored to 
participate in off-site training, particularly through the Healing the Heart 
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of Diversity program of the Fetzer Institute, and the Undoing Racism 
training of the People’s Institute for Survival and Beyond 

Access to complementary anti-racism activities also funded by LSF. For 
example, LSF sponsored a campus tour of the film SKIN DEEP, about 
campus racism. Universities in all of the Project Change sites participated 
in these programs, most in partnership with Project Change. 

Strategic and visible support from LS&Co. staff and management. For 
example, Bob Haas, Chairman of LS&Co., spoke to business leaders at 
anti-racism breakfasts hosted by Project Change in two of the 
communities; 

Feedback and findings from evaluation; and 

Guidance from Foundation staff and coaches hired by the Foundation, and 
since 1 997, support from management at the Project Change National 
Office, a program of the Tides Center. 

The Project Change national office was created by a grant from LSF to the 
Tides Center to establish an ongoing base for the support and expansion of 
anti-racism work, in order to: 

expand the autonomy of Project Change and support its growth 
into a free standing effort; 

- augment incentives and structures for other funders to support the 
work; and 

increase the prospects for the institutionalizing of the work begun 
at LSF through development of a strong organizational base. 



Theory of change 

Over time, the Project Change model has evolved into a more fully established theory of change 
about what it takes for a community/foundation partnership to plan, implement and begin to see 
results from its anti-racism work. Figure 1 on the next page illustrates the theory as it stands 
today. It focuses on the key elements of the model and how they play out in a given community 
in the most general sense. 
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EVALUATION CHALLENGES AND APPROACH 
Challenges 

Design and implementation of an evaluation for Project Change had a number of challenges. 
Many of these are familiar to evaluators of other multi-site community/foundation partnerships 
aimed at improving the quality of life in communities. We offer them in the spirit of James 
Comer’s introduction to his discussion of school reform in New Haven, where he said, “in the 
first year, the kids didn’t learn a lot, but we did.” 

Among the key challenges are: 

• There are multiple audiences for evaluation findings (the communities, the Board 
of Directors and management of the Levi Strauss Foundation, the general field of 
anti-racism activists and community builders), each of which needed different 
kinds of information at different levels of specificity to guide decision-making 
and learning. 

• People had an interest in seeing whether or not the work was producing its 
intended results long before tangible results were likely to be produced or would 
be evident in accessible data. 

• Resources for evaluation, though substantial, were limited. It was important to 
design an evaluation that could meet the needs of its audiences without 
overburdening the sites or drawing an inappropriate level of resources away from 
doing the work. 

• Because strategies were being planned by the communities to meet specific 
community opportunities and needs, the strategies were unknown at the time the 
evaluation was being designed and varied across the sites. 

• The Project Change model relies heavily on voluntary labor, usually with no more 
than one full-time paid staff at any given time (and each has experienced staff 
turnover and times without paid staff). Further, new Project Change structures 
were being formed while the volunteer task forces or Boards also did planning 
and implemented strategies. In the beginning, data collection and documentation 
capacities of the sites were very limited, though they grew much stronger over 
time . 2 



2 Several of the communities now draw on their own considerable capacity to tell their stories and document 
their results. The Foundation, communities and the evaluation team made a mutual decision in 1998 that resources 
did not need to be spent on external, site-based evaluation, at least until sites had a chance to implement strategies 
more fully and longer term results might be captured. At the same time, the groups still benefit from guided 
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• Project Change is a community-wide intervention (that is, it is a saturation 
model). In addition, the interventions that Project Change can effect, even when 
they are very sophisticated and well implemented, are small compared to other 
political and economic changes that influence the quality of life in communities. 
Employment patterns have changed, often dramatically, in Project Change 
communities as a result of military base and manufacturing plant closing or 
restructuring. In some communities, patterns of in- and out- migration have 
created significant changes in the power dynamics and demographics of the 
communities. Some have experienced significant changes in leadership of 
government and community institutions. 

Thus, it would be difficult and probably inappropriate (in a technical sense) to try 
to quantify how much of a change in outcomes is a direct result of Project 
Change. At the same time, because Project Change was new to the communities 
and its activities were being documented by the communities and evaluation, it 
can be relatively straightforward to look at cause and effect through qualitative 
interviews and observations at different points over time. 

There are also some special and important challenges because this is an evaluation of anti-racism 

work. Key among them are: 

• Uncertainty about what short-term markers of progress actually predict lasting 
reduction in personal prejudice, institutional racism or their effects. Even though 
people have been working on reducing racism and its effects for many years (at 
least since slavery and abolition), they continue to be pervasive in the U.S. Thus, 
we are not certain as a field, or as a nation, about what it will take to achieve 
Project Change’s long-term goals. 

This is confounded by the fact that anti-racism work is not linear - disparities and 
injustices sometimes get smaller or fewer but then increase again based on 
political or economic changes in communities or nationally. In fact, one theory is 
that the progress of anti-racism work encounters its strongest resistance just when 
it is most likely to improve the quality of life for a substantial number of people 
or when it most threatens the status quo. This argument is behind the idea of 
“gatekeepers” and why people say that 1) racism, sexism, discrimination based 
on language and immigrant status is more subtle and difficult to address now than 
it was before landmark civil rights legislation, 2) why civil rights are currently 
under legislative and judicial attack and are less aggressively enforced by 
regulatory and policing agencies and 3) why hate crimes and affronts to personal 
liberty are on the rise. 



opportunities for self-reflection and assessment. 
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• Because institutional discrimination is illegal, accurate baseline data on 
institutional policies and practices (e.g., related to reporting and prosecution of 
hate bias crimes, lending policies, assignment of students to gifted programs or 
special education) are closely guarded and difficult to get. Community level data 
showing the effects of institutional racism (income and educational disparities by 
race; patterns of residential segregation) are easier to gather. However, data of 
these kinds are not very timely or sensitive - even substantial and widespread 
change may not show up in the data for several years. 

• There is racism in evaluation, as in other spheres. A key question for evaluation 
of Project Change has been what constitutes success, and who says so. 

Methods, markers of progress and indications of early results 

We adopted the evaluation approach described below given these challenges and the existing 
state of the art in evaluation methodology. The approach evolved considerably over time, as we 
learned more about the nature of community-driven anti-racism work and Foundation support of 
this kind of work, and as we saw what communities were able to accomplish at various stages. 

• The communities and LSF identified a number of markers of progress, often in 
hind-sight, that captured changes in the short-run than might predict changes in 
the quality of life for people in the communities in the long run. In the earliest 
years of Project Change, key markers of progress are: 

- Process goals: the development of adequate levels of trust and analysis 
among the multi-racial task force to plan and implement strategies 
together. Indicators included the ability of task force members to go 
public with the State of Racism research, their willingness to follow- 
through on commitments they made to the group (e.g., to provide access to 
leaders of banking institutions, to bring the media and public scrutiny into 
their work, to develop feasible, detailed and potentially effective strategies 
that were submitted for funding); and their demonstrated ability to stay 
with their work when its goals and efforts were attacked by various 
constituencies important to the group as a whole and its individual 
members. 

- The theoretical sufficiency of planned strategies: There is considerable 
knowledge about what it takes to improve some kinds of outcomes that 
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Project Change sites wanted to effect . 3 This makes it possible to consider 
the likely results of a strategy as it is being planned, well before effects 
can be documented. One early marker of progress for Project Change, 
then, was whether strategies were sufficiently comprehensive in scope and 
scale to produce their intended goals. Sufficiency was measured against 
research and best practices where that information is known. 

Implementation goals: Implementation of key elements of strategies. 
Examples are implementation of anti-racism training with targeted 
stakeholders (e.g., decision makers in lending institutions, the media, 
schools, government, community led organizations, parents of school 
children, heads of faith-based organizations); formation of new and 
apparently action-oriented coalitions (e.g., among law enforcement 
agencies, lending institutions, government agencies); and formation of 
ongoing anti-racism task forces within or across institutions (e.g., within 
the Department of Children and Family Services, community action 
agencies and housing advocates, leadership development groups, on 
college campuses). 

Early results: Indicators include things like changes in the racial 
composition of community boards and commissions that allocate 
resources, as a result of Project Change’s involvement in nominating 
qualified candidates and raising awareness of the importance of this issue; 
changes in the policies and practices of television and newspaper reporting 
on community activities and crime; changes in the way people of color are 
treated when they apply for personal or business loans (whether or not 
they are funneled to non-discriminatory lending instruments or agencies); 
changes in the criteria used to select school officials, teachers, programs 
or curricula; changes in how resources are allocated to improve 
community facilities; changes in the infrastructure of mono-racial 



3 For example, for rates of graduation to change, research indicates that, at a minimum, it takes more 
equitable health outcomes at birth and in early childhood; developmental^ appropriate and unbiased curriculum, 
teaching and promotion policies; attention to learning styles particularly in language development; more equitable 
outcomes that affect drop-out rates and risk behaviors (drinking, early parenting, etc.) and a sense of more equal 
rewards and opportunities for staying in school and good school performance. These are necessary even aside from 
considerations of parent leadership and involvement; uneven resource allocation to schools and other factors known 
to influence school performance. A strategy aimed at improving the equity of graduation rates has to attend to these 
issues, and it has to be implemented for a full cohort of young children from birth through twelfth grade. No one 
program has to do all this, of course, but the community as a whole has to provide the opportunities, activities, 
policies and incentives for a sufficient number of students at a strong enough intensity for it to be likely to occur. 
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organizations and coalitions; and changes in the vocabulary, analysis and attitudes 
of key stakeholders in communities, including government officials, business 
leaders, community advocates and the task force members themselves. 

Data were collected from multiple sources to track these markers of progress. They included 
extensive interviews over multiple times with the same individuals — particularly coalition 
partners, people who were trained and other targets of institutional change; reports from 
community members at forums and focus groups; observation by evaluators; interviews with and 
observations by Foundation staff, TA providers and outside experts (to determine what might be 
meaningful changes); and review of program documents including self-assessments and 
proposals to the Foundation. 

We also attempted to use data collected for the State of Race Relations reports and other public 
and community level data (HMDA and other CRA data, school retention, performance and 
graduation data and police records) to establish a baseline and measure long-term progress. For 
the reasons noted above, we did not rely on changes in these data to measure short-term 
progress. We hope they will be more useful as time passes. But a clear finding for evaluation is 
that these data, even if they can be collected and are accurate, may not show any changes over 
the life of evaluation given the length of time and intensity of strategies it will take to produce 
movement in them. 

Data more sensitive to the kinds of work Project Change is doing will come from within 
institutions, if they will release them. One key is to build institutional accountability, including 
release of relevant data, into the anti-racism agreements that are negotiated with partner 
institutions and coalitions of these institutions. 

Outcome indicators 



Over time, we realized that the evaluation of Project Change is really an exercise in the 
development of grounded theory, where the most meaningful outcome indicators come from 
retrospective review of large quantities of information. That is, we know most clearly what 
matters later, once success is achieved and we can look backwards to see how and why it 
occurred. 

Originally we intended to look at outcomes only in terms of the extent to which the planned 
strategies of Project Change achieved their intended goals, and to the extent these goals fell into 
the categories set by the Foundation for the initiative as a whole. As discussed more fully in the 
findings section, at least one community has achieved results in each of the four goals originally 
set for Project Change: change in institutional policy and practice; prevention of hate crimes; 
improved intergroup or within group relations; and change in the composition of key community 
boards and institutions. Project Change still intends to document these kinds of outcomes as 
work proceeds. 
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However, a mid-point analysis of early results showed that while Project Change was still 
planning and implementing its strategies, it was already producing important benefits for the 
communities in other ways. Thus, we eventually began to capture data on both intended and 
unintended benefits. They fall into the following categories: 

• Intended benefits from strategies (sometimes called proactive benefits): These 
are changes that came from putting in place well-planned, effective strategies, as 
noted above. For example, Albuquerque designed a process to change the 
composition of key boards and institutions. It identified the public and 
public/private boards and their spheres of influence (for example, the zoning 
board influences the ability of businesses to locate in low-income communities); 
researched their current composition, term limits, nomination and selection 
processes; met with representatives of the Mayor to discuss the importance of 
diversity on these Boards; identified well-qualified candidates of color; and 
reached an agreement with city government that Project Change would be among 
the groups notified of vacancies and asked to nominate candidates. They also 
arranged for members of some boards to receive diversity training. Results from 
this strategy are captured as intended benefits. 

• Benefits from good responses to opportunities (sometimes called reactive 
benefits): Many of the Project Change task forces also took organized and 
collective action on issues as they arose. Benefits from this work are captured as 
a good response to opportunity. 

• Watchdog benefits (sometimes called movement benefits): The Project Change 
task forces and staff were the first to call our attention to the limitations of 
assessing their work only from the perspective of results from strategies they 
organized and put in place directly. They told us they were trying to begin, or 
reinvigorate, community-wide anti-racism movements that would have many 
indirect results. 

We thus began to look for, and capture, anti-racism benefits occurring in 
communities that could be traced to the fact that a new multi-racial organization 
existed in the community, was doing visible anti-racism work, and was reaching 
out and connecting with institutions, other activists and organizations. 

For example, in Valdosta, the Ku Klux Klan filed a request for a permit to march 
in the community. Valdosta Project Change heard about the request. Because of 
relationships it had built across the community, it was able very quickly to pull 
people together — including key leadership from many different sectors of the 
community — to discuss how best to oppose the march. When the Klan saw how 
rapidly Project Change and the community could marshal this large and 
influential group, it withdrew its permit request and the march was prevented. 
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As we looked, we found many other examples of these kinds of movement or 
watchdog benefits. 

• Benefits from work that the leadership of Project Change did in spheres outside of 
Project Change (sometimes called leadership benefits). Project Change task 
forces are made up of highly experienced community leaders. 4 In interviews, 
these people reported specific anti-racism actions they had taken that they 
attributed, in whole or in part, to renewed passion and specific training they 
received from Project Change. 

For example, the President of a community college, when negotiating her next 
contract, included specific anti-racism goals for the college to be included in her 
performance based evaluation. A United Way Board member insisted that the 
process of selection for the United Way Executive Director be re-opened and 
done more equitably. The result was appointment of the first Latina United Way 
Executive Director in a community where 70 percent of the population is 
Hispanic. A task force member in a different community helped to introduce 
anti-hate crime legislation in the state senate, where it was eventually approved. 

The evaluation began to capture these activities as another important benefit of 
Project Change. They have turned out to be so prevalent and important that we 
plan to capture them more systematically through a survey of all current and 
former Project Change task force members (discussed more fully in the section on 
Evaluation Next Steps). 

OUTCOMES AND FINDINGS TO DATE 

This section reviews outcomes and findings against two questions of interest to the various 
audiences for evaluation: 

• Have the communities benefitted from their participation in Project Change? 

• Has the initiative been of value as a social investment — that is, has it produced 
benefits beyond the communities where it is being implemented and does it 
further what we know as a field about how to do effective community-driven anti- 
racism work? 



4 Many of them held key community positions - across the four community task forces there were 
community activists, school board members, administrators at colleges and universities, heads of community based 
organizations and non-profit agencies, hospital, law enforcement and business CEOs and managers, clergy, etc. 



Center for Assessment and Policy Development 



Page 14 

Preliminary answers to these questions are highlighted below. 

Benefits to communities to date 



Project Change has produced a number of tangible benefits to the communities in which it has 
been operating. As noted earlier, these benefits fall into four main categories: 

• Results from planned strategies; 

• Benefits from actions Project Change has taken in response to opportunities that 
arose in the communities; 

• Watchdog or movement-like benefits that come from the fact that a multi-racial, 
community-driven anti-racism effort is “on the job” in the community and willing 
to take visible action to stop racist activities or step in to areas of racial tension; 
and 

• From the individual actions of the many community leaders who reported they 
took this action in whole or in part because their anti-racist awareness and skills 
were heightened through their service to Project Change. 

The following pages list substantive results of Project Change for each category for each site. 
Substantive results are those that seem theoretically likely to lead to changes in the quality of life 
for people in the communities and fall into at least one of Project Change’s original four anti- 
racism goals . 5 

In terms of strategies, other key findings are: 

• The most effective strategies are the results of coalitions among Project Change, 
targeted institutions and community advocates. 

• Two strategies that have produced tangible early benefits were implemented 
essentially as designed — the Fair Lending work and the strategy to diversify 
Boards and Commissions in Albuquerque. Except in those instances, strategy 
planning and implementation have taken a lot of trial and error. For example, 
each site began at least one strategy that it decided later was not likely to work 
well enough to be continued. One cannot overestimate how hard this work is, 
and how much we still have to leam about how to negotiate change. 



5 A benefit is listed if it was observed by the evaluation team, noted in the media and/or was confirmed in 
multiple interviews. The tables include benefits that Project Change caused or helped to cause. From among the 
things that occurred in the communities with anti-racism benefits, we attributed to Project Change only those where 
we could confirm the links between Project Change action and the eventual benefit. 
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• The LSF Board of Directors at one point asked whether evaluation evidence 
suggested that communities could have done their strategy work faster, better or 
cheaper. We concluded, given everything that was known at the time the work 
began, they could not. Each of the four multi-racial anti-racism task forces had to 
take the time to do necessary process work and begin to develop an anti-racism 
analysis and vocabulary. All of the institutional partners had to be brought along. 
Trial and error are an inevitable part of the process, as are some level of volunteer 
burn-out and resistance. Their resolution takes time and resources. 

• At the same time, tangible benefits do appear to come more frequently now. Some of 
these benefits are the awaited results of planned strategies. But some are also from 
the increasing capacity of the communities to capitalize on opportunities. So while it 
is not true that early work could be done “faster, better or cheaper,” early investments 
in time and process do make it easier for communities to do better and faster work 
later on. 

Benefits from a social policy perspective 

Project Change has also produced benefits from a social policy perspective. Key findings are: 

• Project Change has demonstrated that tangible anti-racism benefits can be 
produced through multi-racial community driven work. Early results indicate the 
beginnings of success by at least one community in each of the original four 
outcome areas. For example: 

- The banking coalitions in Albuquerque and Valdosta have identified, and 
begun to dismantle, particular institutional policies and practices that 
promote or encourage racial discrimination; 

Public education and coalition building strategies in Knoxville, combined 
with anti-hate crime work, appear to be easing tension between targeted 
members of majority and minority groups; 

The Albuquerque Board and Commission strategy, and action by 
individual task force members (leadership benefits) in El Paso have 
promoted fair representation of community diversity in the leadership of 
important community institutions; and 

- Valdosta Project Change successfully organized its community to prevent 
an overt act of racial prejudice. 
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The work is expanding to other communities, in part because of lessons that 
Project Change has shared. At least three additional communities, and two 
additional funders, used this information to develop anti-racism initiatives that 
may produce similar benefits for other people in other places. 

Project Change has established a national office to create an institutional home for 
anti-racism work of the kind represented by Project Change, support work already 
begun and serve as a catalyst for additional anti-racism work. The office was 
created by a grant from LSF to the Tides Center. 

In addition to its oversight of Project Change and fund raising activities, the 
national office has moved to fill several information gaps in the field. It has 
published in hard copy and on the Internet: 

- A glossary (Attachment I: “ The Power of Words ” Project Change in 
Action, Lessons Learned II) with definitions promoting an analysis of race 
and racism; 

- A resource guide (Anti-Racist Work: An Examination and Assessment of 
Organizational Activity) of trainers, technical assistance providers and 
other materials and resources that Project Change believes meet quality 
standards; 

CAMBIO, a newsletter for the Project Change network that includes 
attention to anti-racism issues as well as updates of the work; 

- A website (anti-racismnet.org) with content that supports anti-racism 
actions in response to current news and learnings; and 

In partnership with The Institute for Democratic Renewal (School of 
Politics and Economics, Claremont Graduate University) the national 
office is producing a guide (Tools for Community Builders ) for 
community collaborations and others on how to attend to personal and 
institutional racism and embed anti-racist practices throughout their 
community building work. 

The office has also helped the Project Change communities to develop and 
upgrade their own information sharing activities. For example, there are 
individual community Project Change websites, linked to a wider audience and 
each other through a national Project Change website (separate from 
antiracismnet.org). 
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The national office also continues to share evaluation findings and lessons with a 
very broad audience of communities, funders, policymakers and advocates, in two 
documents (Lessons Learned I, How to Plan An Anti-Racism Initiative, Lessons 
Learned II, Project Change in Action). 

EVALUATION NEXT STEPS 

We are planning on doing a summative evaluation of Project Change that will be published in 
2002, to look at outcomes and lessons ten years after the initiative began. In the interim, the 
national Project Change office continues to identify gaps for the field which we may help them 
to fill. 

For example, there is a demand from communities and funders for more detailed and evaluative 
information about available training and technical assistance resources and its results. Project 
Change particularly wants to let people know what it takes to support anti-racism work - as 
distinct from issues of diversity, multiculturalism, etc. They would also like people to know 
more about the scope and outcomes of their investment in multiple kinds of training and 
leadership development for Project Change communities. CAPD is likely to be working with 
Project Change to document and assess results from these kinds of activities. 

It has also become clear to us, over the course of our work with Project Change, Healing the 
Heart of Diversity, and development of the Democracy, Race, Community Building Handbook 
that the state of the art in evaluation of anti-racism efforts lags behind what programs and 
community collaborations need to track their progress and tell their stories. As a field, we need 
to share what we have learned about evaluating these kinds of programs, in order to: 

• Understand and articulate different theories of change that drive the work; 

• Identify sound ways to know if programs are on track to achieving their long-term 
anti-racism goals; 

• Create benchmarks for process and implementation work, in terms of the time it 
takes, necessary and sufficient components, and resources; and 

• Set high but realistic goals that might promote additional long-lasting and 
effective work. 

CAPD is currently developing a project to help upgrade the collective capacities of anti-racism 
programs, evaluators and funders to meet these goals. 
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